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Abstract — The method of a linear high dynamic range imaging using solid-state photosensors with Bayer colour filters 
array is provided in this paper. Using information from neighbour pixels, it is possible to reconstruct linear images with 
wide dynamic range from the oversaturated images. Bayer colour filters array is considered as an array of neutral filters 
in a quasimonochromatic light. If the camera's response function to the desirable light source is known then one can 
calculate correction coefficients to reconstruct oversaturated images. Reconstructed images are linearized in order to 
provide a linear high dynamic range images for optical-digital imaging systems. 

The calibration procedure for obtaining the camera's response function to the desired light source is described. 
Experimental results of the reconstruction of the images from the oversaturated images are presented for red, green, and 
blue quasimonochromatic light sources. Quantitative analysis of the accuracy of the reconstructed images is provided. 
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1 Introduction 

Growing interest to the high dynamic 
range (HDR) imaging is observed recently. 
Such synthesized images from a set of the 
pictures with different exposure times are char- 
acterised by wide dynamic range. Numerous 
reconstruction and tone mapping methods [1Q, 
fsl] were proposed for digital HDR photog- 
raphy. 

But HDR imagery is suitable not only for 
photography. High dynamic range images are 
required for applications such as automotive 
systems 0], vision systems and hy- 

brid optical-digital imaging systems based on 
"wavefront coding" paradigm |@], flj^], |@]. 

The aim of this article is to show that us- 
ing conventional solid-state photosensors with 
Bayer [jlG] colour filters array it is possibly 
to recover linear high dynamic range images. 
Because of using a quasimonochromatic light 
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in hybrid optical-digital imaging systems, in 
this paper we are dealing with LEDs for the 
input scene illumination. 

Linear high dynamic range images can be 
constructed using Spatially Varying pixel Ex- 
posures (SVE) technique, proposed in 111)] , B120. 
This technique allows to construct high dy- 
namic range images using information from 
the neighbour pixels. When a pixel is satu- 
rated in the acquired image, it is likely to 
have a neighbour pixel that is not. Analysing 
the neighbour pixel's values, it is possible to 
construct a high dynamic range image. Such 
image is non-linear, hence linearization of the 
constructed SVE image is necessary. Lineariza- 
tion of a constructed SVE image is performed 
using correction coefficients that are obtained 
at the preliminary stage of calibration. 

The calibration procedure for obtaining of 
the camera's response funstion to the light- 
source is provided. Experimental results of the 
oversaturated images reconstruction using the 
SVE technique are presented. 
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2 Linear high dynamic range imag- 
ing 

First of all, the camera's response function to 
the desirable lightsource should be obtained. A 
flat-field scene in a quasimonochromatic light 
is taken with different exposures times for ob- 
taining of camera's response function. During 
such calibration process, the camera's response 
to the lightsource as well as correction coeffi- 
cients for SVE-images are calculated. 

If the correction coefficients are evaluated 
then oversaturated images, which are cap- 
tured in the same light, can be reconstructed. 
Oversaturated images are analysed and sat- 
urated pixels are replaced using information 
from the neighbour pixels. Such constructed 
SVE-images are characterised by 7-like non- 
linearity. Then constructed SVE-images are lin- 
earized using correction coefficients, which 
were obtained at the stage of calibration. Re- 
constructed linear image is characterised by 
wide dynamic range and a linear response to 
the registered light. 

2.1 Preliminary calibration procedure 

The procedure of calibration is required for 
obtaining of the correction coefficients for the 
SVE images to be linearized. For such pur- 
pose, a light from the lightsource is passed 
through opal glass for elimination of the flat- 
field non-uniformity. Pictures of the flat-field 
are captured with different exposure times. In 
this work were used red, green, and blue LEDs 
driven with DC current. 

Main pixels are chosen accordingly to the 
dominant lightwave of a lightsource; then first 
and second extra pixels are selected. Each cap- 
tured picture of the flat-field is decomposed to 
the three images according to Bayer's RGGB 
colour matrix. Denote by I ma in(hj) the main 
pixels array, h(i,j) the first extra pixels ar- 
ray, hihj) the second extra pixels array. It is 
needed to construct an HDR image for each 
picture. For the HDR image to be constructed 
properly it is necessary to determine the end of 
linearity (EOL) value for the camera's sensor. 
Used in this work camera is characterized by 
EOL value 3400 DN for 12-bit sensor. 



The algorithm for construction of the high 
dynamic range image is: 

1. If Imain{i,j) > EOL, Then 
Imain(i,j) = EOL + h(i,j). 
1.1. If h(i,j) > EOL, Then 

Imainihj) = 2 ' EOL + I 2 (i, j). 

2. If Imainihj) < EOL, Then 

As a result, non-linear HDR images can be 
constructed using such algorithm. Using de- 
scribed algorithm at the stage of calibration, the 
radiometric function can be obtained. 

For the analysis and evaluation of the radio- 
metric function, pictures of flat-field are taken 
for different exposures times. Then obtained 
images of the flat-field are constructed into 
HDR image using described above algorithm. 
An area of 256 x 256 pixels of such image 
is selected, mean value of such area is cal- 
culated. Thus radiometric function, which is 
mean signal's value versus relative exposure 
time dependency, can be obtained. For exam- 
ple, the radiometric function for red LEDs used 
in this work is presented in Fig. [0 Further, 
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Fig. 1 . The effective radiometric function of the 
SVE imaging system. 

the correction coefficients must be calculated in 
order to compensate non-linearity of the SVE 
imaging system. The linear part of the radio- 
metric function is fitted to a line aT+b, where T 
is an exposure time (see Fig. |2]). The accuracy of 
fitting a line to the experimental data is signif- 
icant: slight deviation of a line produces great 
errors on the reconstructed images. The Trust- 
Region [13Q, |14(1 fitting algorithm was used 
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because of speed and acceptable accuracy. Sev- 
eral limiting factors were specified, for example 
coefficient b should be the smallest possible 
(when there is no light, no signal should be 
detected). 
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Fig. 2. The approximation of the radiometric 
function: a line is fitted to the linear part of the 
data, and a high-order polynomial is fitted to the 
non-linear part of the data. 

The high-order polynomial model proposed 
by Mitsunaga [15] was used for approximation 
of the radiometric function's non-linear part: 



f(T) 



N 
n=0 



(1) 



As a result of the calibration, unknown coeffi- 
cients p n are determined from the experimental 
data. Thus the correction coefficients can be 
calculated: 



aT + b 



Q's 



N 

E Pr, 

n=0 



(2) 



Using Eq|2] for calculation of the correction 
coefficients a sve it is possible to compensate 
SVE imaging system's non-linearity. 

2.2 Reconstruction of the SVE image 

Main principle of the SVE technique according 



to lUD, 1121 ] is to simultaneously sample the 
spatial and exposure dimensions of the image 
irradiance. An array of neutral filters on a pho- 
tosensor can be used to detect different levels 



of the input scene's signal. On the captured im- 
age, the brighter pixels have greater exposure 
to image irradiance and the darker ones have 
lower exposure. Thus the registered image con- 
tains information about the input scene signal 
as well as the exposure time information. 

In most conventional digital cameras, 
Bayer's colour filters array is applied. In a 
quasimonochromatic light Bayer matrix can be 
considered flit] as an array of neutral filters 
(see Fig. [3}. Hence it is possible to apply 
SVE technique for increasing of the image's 
dynamic range. 

Denote by "main pixels" all pixels under fil- 
ters of the colour same as illumination's colour. 
If main pixels are saturated, it is likely to 
have neighbour pixels that are not. Denote by 
"first extra pixels" and "second extra pixels" 
neighbour pixels under different colour filters. 
For example, if the main pixels are red then first 
extra pixels are green and second extra pixels 
are blue. 
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Fig. 3. Bayer mosaic in a quasimonochromatic 
light can be considered as an array of neutral 
filters. 

The algorithm for SVE image construction is 
the same as for calibration (see Subsection 12. 
To compensate 7-like non-linearity of such SVE 
imaging systems, the linearization of the SVE- 
images is required. 

For linearization of the SVE image, correc- 
tion coefficients must be used. Such coefficients 
were calculated at the preliminary stage of cal- 
ibration, as it mentioned in Subsection I2.1[ The 
problem is that there are only a few correction 
coefficients that conforms to non-linear data 
points. In other words, for the SVE image to 
be linearized there are insufficient amount of 
correction coefficients. 

The polynomial model is used for interpo- 
lation of the correction coefficients in order 
to overcome such problem. High-order poly- 
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nomial is fitted to the data obtained at the 
calibration stage. Thus an unknown correction 
coefficient can be calculated for almost any 
non-linear data value of the SVE constructed 
image. 

It is significant to estimate the accuracy of the 
reconstructed images due to complexity of the 
reconstruction process. The quantitative results 
of the reconstruction and linearization of the 
SVE images are provided below. 



3 Experimental results 

The high dynamic range scene was created for 
the optical experiments. The photo of the test 
scene is presented in Fig [4] (image is scaled 
down to 8-bit and contrasted for publishing). 
Scene's background is a light-absorption fabric, 
and the test image is illuminated by LED lamp. 

The properties of the lightsources used in 
this work as well as transmittance coefficients 
are described in Table [H It should be noted 
that transmittance coefficients for Bayer mosaic 
are obtained for used in this work commercial 
digital camera Canon EOS 400D. 



mode" with command dcraw -4 -D -T file- 
name.crl. This command produces an unpro- 
cessed ("totally raw") linear 12-bit TIFF images. 



Parameters 


Main pix. 


cf 

1 extra pix. 


2 nd extra pix. 




Ad = 


= 625 nm 




Pixels order 


red 


green 


blue 


Transmittance 


ei = 1.00 


e 2 = 0.20 


e 3 = 0.09 




Ac = 


= 520 nm 




Pixels order 


green 


blue 


red 


Transmittance 


ei = 1.00 


e 2 = 0.33 


e 3 = 0.15 




Ac = 


= 470 nm 




Pixels order 


blue 


green 


red 


Transmittance 


ei = 1.00 


e 2 = 0.45 


e 3 = 0.08 



TABLE 1 

Transmittance coefficients of Bayer-covered 
photosensor for different LEDs used in this 
work. 



The test image consists of binary graphics, 
periodical elements, textual elements of dif- 
ferent size, and gradient bars. Gradient bars 
are used for the estimation of the halftone 
stability of the reconstructed images. The test 
image was captured by the digital camera with 
an exposure time varied from 1/4000 to 2 
seconds. All captured images were processed 
by DCRAW |17|1 converter in the "document 




Fig. 4. The test scene with high dynamic range 
registered by the photo sensor in the red LEDs 
illumination. 

If an exposure time is short then a digital 
signal's value of the captured image is lesser 
than camera's ADC limit. In this case the val- 
ues of the captured image are unchanged. The 
images captured with long exposure time are 
oversaturated in the main colour. Thus using 
information from the neighbour pixels it is pos- 
sible to reconstruct such oversaturated image. 
The SVE reconstruction algorithm is the same 
as mentioned in Section 12.11 

Selecting only main pixels from the original 
image ("red" pixels because of red illumina- 
tion), one can obtain a new image. Let us 
denote such image as "original image" in order 
to calculate the NRMS error. 

As above, using SVE reconstruction method 
and obtained calibration coefficients it is possi- 
ble to reconstruct a linear high dynamic range 
image from the oversaturated one. The whole 
size of the reconstructed linear SVE-image is 
1953 x 1301 pixels, the informative part is 300 x 
300 pixels. 

It is significant to estimate the halftone's sta- 
bility of the SVE-reconstructed image. On the 
SVE-reconstructed image, the gradient bar with 
8 halftone was analysed and 8 areas were se- 
lected in this gradient bar. Each size of the area 
was 15 x 15 pixels. To estimate the halftones 
stability, relations between the mean value of 
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each area and the mean value of the "white" 
area (the brightest area) were measured. This 
operation was applied to every reconstructed 
image. 

3.1 Red LEDs illumination 

Test scene was illuminated by red LEDs with 
dominant wavelength X D = 625 nm. Main 
pixels for such illumination are red, first extra 
pixels are green, and second extra pixels are 
blue. As it was mentioned in Table [TJ, transmit- 
tance coefficient for green colour filters is 0.20 
and for blue colour filters is 0.09. One of the 
reconstructed image is presented in Fig. |5j 
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Fig. 5. The reconstructed linear SVE image 
taken by digital camera in a quasimonochro- 
matic red light. 

Reconstructed images using only first ex- 
tra pixels are characterised by linear dynamic 
range of 71-84 dB and the NRMS error between 
the original image and reconstructed images 
of 5-10% (see Fig. [6). Such NRMS error is 
considered as acceptable for practical applica- 
tions in optical-digital imaging systems. For the 
reconstruction process there were used around 
87% of first extra pixels. 

Using first and second extra pixels it is 
possible to reconstruct images with dynamic 
range of 87-95 dB. The NRMS error between 
the original image and reconstructed images 
is around 11-15%. There were used 96-98% of 



5 
DC 



70 75 80 85 

Dynamic range of the reconstructed images, dB 



Fig. 6. For red LEDs illumination: NRMS error 
between the original image and reconstructed 
images versus dynamic range. 



the first extra pixels and 65-70% of the second 
extra pixels to reconstruct such oversaturated 
images. Results of the NRMS error are sum- 
marized in FigJSJ 
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60 65 70 75 80 85 90 95 

Dynamic range of the reconstructed images, dB 

Fig. 7. The halftones stability of the SVE- 
reconstructed images for red LEDs light. 

The halftone stability of the reconstructed 
images was evaluated as well. From Fig. it 
can be noted that images with dynamic range 
more than 84 dB are characterised by less stable 
halftone relations. Instability of the halftone 
relations in the range of 85 to 90 dB can be ex- 
plained by transition to the second extra pixels 
usage. It also should be noted that halftones 
on the red-illuminated images are more dense, 
i.e., recovered image became darker than the 
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original one. Experimental results for red LEDs 
illumination are summarized in Table |2j 



Parameters 


Main 
pixels 


1 st extra 
pixels 


2 nd extra 
pixels 


Dynamic range 
NRMSe 
Pixels used 


< 70 dB 

< 5% 

< 1% 


71-84 dB 

5-10% 

< 87% (1 st ) 


87-95 dB 
11-15% 

< 95% (1 st ) 

< 81% (2 nd ) 



TABLE 2 

Quantitative results for SVE reconstructed 
images with red LEDs illumination. 



It is clear that dynamic range of the images 
in a quasimonochromatic red light can be in- 
creased using SVE technique up to 95 dB with 
acceptable (about 15%) NRMS error from the 
original image and stable halftone relations on 
the recovered image. 

3.2 Green LEDs illumination 

The same test scene was illuminated by green 
LEDs with \d = 520 nm dominant wavelength. 
Main pixels for such illumination are green, 
first extra pixels are blue, and second extra pix- 
els are red. According to Table CO the coefficient 
of transmittance for blue colour filters is 0.33 
and for red colour filters is 0.15. 

It should be noted that the choice of extra 
pixels is ambiguous for green light illumina- 
tion because of transmittance characteristics of 
the colour filters. When the spectrum of green 
lightsource is more "cold" then first extra pixels 
should be blue and second ones should be 
red. Otherwise when the spectrum of green 
lightsource is more "warm", it is better to 
assign first extra pixels as "red" and second 
ones as "blue". It is advisable to preliminary 
estimate the transmittance of green light from 
desired lightsource for red and blue colour 
filters. Wrong choice of the extra pixels can lead 
to halftone degradation on the reconstructed 
images. 

Last but not the least, it is hard to obtain 
wide dynamic range reconstructed images in 
green light due to similar coefficients of trans- 
mittance for red and blue colour filters. 

Reconstructed images using only first extra 
pixels (blue in this case) are characterised by 
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Fig. 8. For green LEDs illumination: NRMS error 
between the original image and reconstructed 
images versus dynamic range. 

linear dynamic range of 75-80 dB and the 
NRMS error between the original image and 
reconstructed images of 3-7% (see Fig. ®. Less 
than 55% of first extra pixels was used for 
reconstruction. 




16pt18pt 20pt^ 

Fig. 9. Reconstructed image in green LEDs 
illumination with wrong halftone relations. 

But when second extra pixels are used 
there are observed significant NRMS error and 
halftones destabilisation (see Fig. |9] and Fig.[T0|). 
Although the dynamic range of such recon- 
structed images is more than 85 dB, the NRMS 
error is 20-35%. Thus for the green light is 
needed more sophisticated algorithm in order 
to provide better images stability. As it men- 
tioned above in this subsection, it is difficult to 
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achieve a linear dynamic range of the images 
more than 85 dB in a green light. 
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Fig. 10. The halftones stability of the SVE- 
reconstructed images for green LEDs light. 



Parameters 


Main 
pixels 


1 st extra 
pixels 


2 nd extra 
pixels 


Dynamic range 
NRMSe 
Pixels used 


< 70 dB 

< 7% 

< 1% 


75-80 dB 
3-7% 

< 55% (1 st ) 


84-86 dB 
7-38% 

< 91% (1 st ) 

< 72% (2 nd ) 



TABLE 3 

Quantitative results for SVE reconstructed 
images with green LEDs illumination. 



Images were reconstructed using only first 
extra pixels (green in this case). Reconstructed 
images are characterised by linear dynamic 
range of 70-88 dB and NRMS error between 
the original image and reconstructed images of 
9-15%. Such NRMS error is large enough and 
may lead to degradation of the reconstructed 
image. In Fig [TTJ is presented recovered image 
with bright spots (probably due to parasitic 
reflection from the laser printer's toner of the 
printed test image). Less than 58% of first extra 
pixels were used for the reconstruction. 




Fig. 11. Reconstructed image in blue LEDs 
illumination. 



Obtained experimental results for green 
light, which are summarized in Table [3j allow 
to argue that using SVE technique it is possi- 
ble to reconstruct oversaturated images to lin- 
ear high dynamic range images with dynamic 
range up to 80 dB and NRMS error less than 
7%. However further increasing of dynamic 
range is required more sophisticated algorithm 
for image's reconstruction. 

3.3 Blue LEDs illumination 

Test scene was illuminated by blue LEDs with 
dominant wavelength of A^> = 470 nm. Main 
pixels for such illumination are blue, first extra 
pixels are green, and second extra pixels are 
red. As it mentioned in Table [TJ coefficient of 
transmittance for green colour filters is 0.45 and 
for red colour filters 0.08. 



60 65 70 75 80 85 90 95 

Dynamic range of the reconstructed images, dB 

Fig. 1 2. For blue LEDs illumination: NRMS error 
between the original image and reconstructed 
images versus dynamic range. 

Using first and second extra pixels it is 
possible to reconstruct images with dynamic 
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range of 90-95 dB. The NRMS error between 
the original image and reconstructed images is 
around 11-18% (see Fig. |T2|) . There were used 
94% of the first extra pixels and 88% of the 
second extra pixels to reconstruct such over- 
saturated images. From Fig. [13] it can be noted 
that images with dynamic range more than 
88 dB are characterised by less stable halftone 
relations. 
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Fig. 13. The halftones stability of the SVE- 
reconstructed images for blue LEDs light. 

Further increase of dynamic range of the 
images beyond 90 dB using the secondary 
extra pixels produces images with less stable 
halftone relations and bigger NRMS error from 
the original image. 



Parameters 


Main 
pixels 


1 st extra 
pixels 


2 nd extra 
pixels 


Dynamic range 
NRMSe 
Pixels used 


< 70 dB 

< 10% 

< 1% 


70-88 dB 

9-15% 

< 58% (1 st ) 


90-95 dB 
11-18% 

< 94% (1 st ) 

< 88% (2 nd ) 



TABLE 4 

Quantitative results for SVE reconstructed 
images with blue LEDs illumination. 



4 Conclusion 

The reconstruction accuracy and stability of 
the reconstructed linear high dynamic range 
images using SVE technique are discussed in 
this paper. It is shown the possibility of the 
linear HDR imaging in a quasimonochromatic 
light using Bayer-covered photo sensor and the 
SVE technique. 

For the red LEDs illumination it was ob- 
tained that SVE-reconstructed images are char- 
acterized by dynamic range up to 84 dB using 
first extra pixels and NRMS error between the 
original image and reconstructed images up to 
10%. Using both first and second extra pixels 
it is affordable to reconstruct a linear HDR 
image with dynamic range of 90-95 dB but with 
greater NRMS error of 11-15%. 

Results for the blue light illumination are 
similar to the results for red LEDs. Applying 
first extra pixels to restore image provides lin- 
ear dynamic range of the images up to 88 dB 
and NRMS error up to 15%. For both first 
and second extra pixels linear dynamic range 
increases to 90-95 dB as well as NRMS error of 
11-18%. 

Reconstructed images for the green light il- 
lumination showed lower dynamic range: only 
75-80 dB for first extra pixels and NRMS er- 
ror up to 7%. Further increasing of dynamic 
range leads to image degradation and unac- 
ceptable NRMS error. Hence for the green light 
is needed more sophisticated algorithm in or- 
der to provide better images stability. 

Provided quantitative analysis of the recon- 
structed images allows to say that the accuracy 
of the method is enough for practical applica- 
tions. Such applications can be optical-digital 
imaging systems based on "wavefront coding" 
paradigm, in-vehicle systems of images under- 
standing, and edge detection systems. 



It can be noted that using first extra pixels 
one can reconstruct oversaturated images to 
linear high dynamic range images with dy- 
namic range up to 88 dB and NRMS error 
less than 15% (see Table H). Increasing dynamic 
range using first and second extra pixels can 
produce images with less stable halftone. 
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